Papers with representation quality
RAVEN: Query-Guided Representation Alignment for Question Answering over Audio, Video, Embedded Sensors, and Natural Language (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal question answering often requires identifying which video, audio, or sensor tokens are relevant to the question. off-camera speech, background noise, or motion outside the field of view often mislead fusion models that weight all streams equally. |
| Approach: | They propose a unified architecture for multimodal question answering that assigns scalar relevance scores to each token across modalities. |
| Outcome: | The proposed model outperforms state-of-the-art multimodal large language models on seven multi-modal QA benchmarks and egocentric and exocentric tasks. |
Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to commonsense inference lack coverage and expressive diversity of commonsensense knowledge graphs. |
| Approach: | They propose a framework that contrasts sets of semantically similar and dissimilar events . they propose 'solar' framework that can be used to learn commonsense inference . |
| Outcome: | The proposed framework outperforms the state-of-the-art commonsense transformer on commonsensense inference by 1.84% on average among 8 metrics. |
DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing multimodal sentence representation learning methods focus on aligning images and text at a coarse level, resulting in cross-modal misalignment bias and intra-modal semantic divergence. |
| Approach: | They propose a dual-level alignment learning framework for multimodal sentence representation learning that promotes cross-modal and intra-modal alignment. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on semantic textual similarity and transfer tasks on semantic similarity, ranking distillation and global intra-modal alignment learning. |
EEE-QA: Exploring Effective and Efficient Question-Answer Representations (2024.lrec-main)
Copied to clipboard
| Challenge: | Current approaches to question answering rely on pre-trained language models like RoBERTa. |
| Approach: | They propose a pooling approach that embeds all answer candidates with the question . they also propose enabling cross-reference between answer choices . |
| Outcome: | The proposed methods improve throughput and memory efficiency with little sacrifice in performance. |
Contrastive Deterministic Autoencoders For Language Modeling (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Variational autoencoders (VAEs) are a popular family of generative models with wide applicability. |
| Approach: | They propose to modify a deterministic model designed for images to avoid posterior collapse by controlling the entropy of the aggregate posterior to make it Gaussian. |
| Outcome: | The proposed models outperform a broad range of VAE models on text generation and downstream tasks from representations while avoiding reparametrization steps. |
HiCOT: Improving Neural Topic Models via Optimal Transport and Contrastive Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in neural topic models (NTMs) have improved topic quality but still face challenges: weak document-topic alignment, high inference costs due to large pretrained language models, and limited modeling of hierarchical topic structures. |
| Approach: | They propose a framework that integrates hierarchical clustering and contrastive learning to refine document-topic relationships using compact PLM-based embeddings. |
| Outcome: | The proposed framework improves topic coherence, topic performance, representation quality and computational efficiency over existing NTMs. |
ConPrompt: Pre-training a Language Model with Machine-Generated Data for Implicit Hate Speech Detection (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing pre-trained language models for hate speech detection are not specialized in implicit hate speech. |
| Approach: | They propose a pre-trained language model for implicit hate speech detection that leverages machine-generated data to train the model. |
| Outcome: | The proposed model can be trained on a massive hate speech dataset with positive samples . it can be generalized and reduce identity term bias, the authors show . |
Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings (2026.acl-long)
Copied to clipboard
Xueying Ding, Xingyue Huang, Mingxuan Ju, Liam Collins, Yozen Liu, Leman Akoglu, Neil Shah, Tong Zhao
| Challenge: | Large language models produce powerful text embeddings, but their causal attention mechanism restricts the flow of information from later to earlier tokens, harming performance. |
| Approach: | They propose a method that prepending a single summary token to reduce attention-level compression by partitioning the input into blocks and prepending blocks to subsequent blocks. |
| Outcome: | The proposed method achieves consistent performance gains across 11 retrieval datasets and 30 general embedding benchmarks. |
Distance-Based Propagation for Efficient Knowledge Graph Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Knowledge graph completion (KGC) aims to predict unseen edges in knowledge graphs (KGs) . a few recent attempts to address this problem sacrifice the performance to gain efficiency. |
| Approach: | They propose a method that aggregates path information to solve this problem by aggregating paths in a fixed window for each source-target pair. |
| Outcome: | The proposed method can cut down on the number of propagated messages by 90% while achieving competitive performance on multiple KG datasets. |
Domain Pre-training Impact on Representations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a small, specialized corpus can produce effective representations, but the quality of pre-training is not affected by the choice of corpus. |
| Approach: | They focus on the representation quality achieved through pre-training alone . |
| Outcome: | The results show that pre-training on a small, specialized corpus produces effective representations. |
Language Fusion for Parameter-Efficient Cross-lingual Transfer (2025.acl-long)
Copied to clipboard
| Challenge: | Limited availability of multilingual text corpora for pretraining results in poor performance on downstream tasks due to undertrained representation spaces for languages other than English. |
| Approach: | They propose a method that integrates source and target language representations within low-rank (LoRA) adapters using lightweight linear transformations to enhance representation quality and transfer performance for languages other than English. |
| Outcome: | The proposed method improves representation quality and performance for languages other than English while maintaining parameter efficiency. |
SentiCSE: A Sentiment-aware Contrastive Sentence Embedding Framework with Sentiment-guided Textual Similarity (2024.lrec-main)
Copied to clipboard
| Challenge: | Sentiment-aware pre-trained language models (PLMs) demonstrate impressive results in downstream sentiment analysis tasks, but they neglect to evaluate the quality of constructed sentiment representations. |
| Approach: | They propose a new metric for evaluating the quality of sentiment representations that is based on the degree of equivalence in sentiment polarity between two sentences. |
| Outcome: | The proposed framework outperforms the existing sentiment-aware models in sentiment analysis tasks. |